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FOREWORD 


The  principal  mission  of  the  Training  Research  Laboratory  of  the  Army 
Research  Institute  is  to  refine  the  Army's  training  methods  and  practices. 
As  part  of  this  refinement,  the  ARI-Fort  Knox  Training  Technology  Field 
Activity's  (TTFA)  mission  is  to  develop,  evaluate,  and  implement  new  tech¬ 
nologies  and  methods  into  Army  training.  This  TTFA's  recent  efforts  have 
focused  on  the  potential  of  computer  and  videodisc-based  technologies  for 
Armor  training. 

This  report  provides  a  description  and  evaluation  of  an  initial  TTFA 
product,  computer  and  videodisc-based  courseware  for  training  land  naviga¬ 
tion  skills.  Successful  implementation  of  this  courseware  in  the  Ml  tank 
commanders  course,  19K  BNCOC,  would  support  the  transfer  of  automated 
training  technologies  into  the  Army  training  system. 

/ 

EDGAR  M.  JOHNSON 
Technical  Director 


LAND  NAVIGATION  SKILLS  TRAINING:  AN  EVALUATION  OF  COMPUTER  AND 
VIDEODISC-BASED  COURSEWARE 


EXECUTIVE  SUMMARY 


Requirement : 

To  evaluate  the  effectiveness  of  the  computer  and  videodisc-based 
courseware  developed  for  training  basic  land  navigation  skills. 


Procedure : 

As  part  of  the  Training  Technology  Field  Activity  (TTFA)  mission  to 
provide  innovative  training  technologies  for  Army  training,  computer  and 
videodisc-based  instruction  was  developed  for  training  and  testing  basic 
land  navigation  skills.  The  training  effectiveness  of  this  courseware  was 
evaluated  with  a  sample  of  44  military  personnel  similar  to  soldiers  in 
the  Ml  tank  commanders  course,  19K  Basic  Noncommissioned  Officer  Course 
(BNCOC).  All  training  and  tests  were  delivered  on-line  by  a  MicroTICCIT 
System  II  computer  delivery  system.  A  pretest-posttest  design  with  sev¬ 
eral  control  conditions  was  used  to  determine  the  courseware’s  training 
effectiveness. 


Findings : 

The  courseware's  training  effectiveness  for  three  of  the  tasks — 
Identify  Natural  Terrain  Features,  Locate  an  Unknown  Point  on  the  Ground 
by  Intersection  or  Resection  and  Analyze  Terrain  Using  the  Five  Military 
Aspects  of  Terrain — was  demonstrated  by  significant  improvements  in  par¬ 
ticipants'  posttest  scores,  and  an  average  increase  for  the  sample  from 
pretest  "NO  GO"  to  posttest  "GO."  In  addition,  the  findings  demonstrated 
that  the  courseware  was  more  efficient  than  the  conventional  instruction 
and  user-friendly. 

For  the  other  two  tasks — Orient  a  Map  and  Determine  Location  by  Map 
Terrain  Association — the  design  of  the  courseware  was  found  inadequate 
and  design  modifications  were  recommended. 


Utilization  of  Findings: 

The  findings  of  this  evaluation  were  provided  to  the  Technical  Direc¬ 
tor  of  the  Armor  School  who  recommended  validation  of  the  courseware  found 
effective  in  the  1 9 K  BNCOC  program  of  instruction.  Successful  Implementa¬ 
tion  into  this  course  would  support  the  transfer  of  this  courseware  to 
other  Army  training  systems. 
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LAND  NAVIGATION  SKILLS  TRAINING:  AN  EVALUATION  OF  COMPUTER  AND 
VIDEODISC-BASED  COURSEWARE 


INTRODUCTION 

While  the  Army  training  system  has  incorporated  numerous  new  technol¬ 
ogies  in  recent  years,  it  is  still  heavily  dependent  upon  approaches  from 
an  earlier  era,  such  as  use  of  paper  materials  and  classroom  lecturing 
techniques.  In  an  operational  training  environment  it  is  difficult  to 
systematically  test  and  apply  promising  technologies  and  innovations, 
especially  given  the  rapid  rate  at  which  technologies  are  being  developed 
today. 

To  overcome  this  difficulty,  the  U.S,  Army  Training  and  Doctrine  Com¬ 
mand  (TRADOC),  the  US  Army  Armor  Center  (USAARMC),  and  the  US  Army  Re¬ 
search  Institute  (ARI)  have  established  the  Training  Technology  Field 
Activity  (TTFA)  at  Fort  Knox,  Kentucky.  The  mission  of  the  TTFA  is  to 
systematically  identify,  evaluate,  and  introduce  new  training  methods, 
techniques,  technologies,  and  models  to  Army  training.  A  more  complete 
discussion  of  TTFA's  background,  approach  and  scope  of  effort  is  provided 
by  Kristiansen  (1986). 

The  initial  efforts  of  the  Fort  Knox  TTFA  have  focused  on  the  Basic 
Noncommissioned  Officer  Course  (BNCOC)  training  program  for  Ml  tank  conr- 
manders  with  the  Military  Occupational  Speciality  (MOS)  19K.  A  review  of 
BNCOC  performance  records  and  interviews  with  BNCOC  instructors  identified 
land  navigation  as  one  of  the  most  difficult  training  problems  in  the 
BNCOC  program  of  instruction  (POI).  This  paper  reports  the  development 
and  evaluation  of  the  TTFA'S  courseware  for  this  block  of  Instruction, 
land  navigation  skills  training. 


Background 

The  difficulties  inherent  in  land  navigation  training  and  performance 
are  not  recent  and  certainly  not  limited  to  the  area  of  Armor  operations. 
In  the  late  1950's,  the  Army  commissioned  a  series  of  land  navigation  re¬ 
search  efforts  which  were  conducted  at  Fort  Knox  with  Armor  personnel 
(Findlay,  Roach,  and  Logan,  1957;  Logan,  Willmorth,  and  Findlay,  1957; 
McGuigan,  1957;  Willmorth  and  Logan,  1957).  This  work  resulted  in  a  number 
of  useful  recommendations  for  the  training  of  land  navigation  skills — but 
a  conclusion  that  land  navigation  Is  a  very  difficult  task  to  train,  and 
skill  to  acquire.  For  example,  out  of  96  soldiers  who  had  completed  basic 
combat  training  and  received  a  two  hour  review  of  map  reading  skills  be¬ 
fore  testing,  83  percent  failed  half  of  their  six  field  navigation  mis¬ 
sions  and  17  percent  failed  on  all  six  missions  tested  (Findlay  et  al. 
(1957). 

Later  and  in  a  different  training  setting,  infantry  soldiers  at  Fort 
Benning,  50  percent  failed  to  reach  their  objective  over  terrain  routes 
rated  as  "easy”  to  “moderate"  and  only  52  reached  the  objective  when 


navigating  over  "difficult"  routes  (Powers,  1964).  While  field-based 
research  in  land  navigation  has  drastically  declined  in  the  last  two  dec¬ 
ades,  a  review  of  the  more  recent  literature  reveals  little  improvement  in 
the  performance  of  land  navigation  tasks  by  military  personnel  (Cross, 
Rugge,  and  Thornydke,  1982;  Pleban  and  Grainer,  1985). 


APPROACH 


Component  Skills 

The  courseware  in  question  focused  on  a  subset  of  the  basic  skills  re¬ 
quired  for  tactical  land  navigation.  A  more  complete  set  of  the  component 
skills  required  for  land  navigation  is  provided  by  Findlay  et  al.  (1957) 
and  Powers  (1964).  Tasks  included  in  the  current  courseware  were  selected 
by  the  Land  Navigation  Steering  Committee.  This  committee  was  established 
by  the  TTFA  to  ensure  that  the  courseware's  training  objectives  were  coor- 
dinated  with  the  appropriate  military  training  and  doctrine  agencies.  The 
committee  included  designated  representatives  from  the  NCO  Academy,  the 
Directorate  of  Training  and  Doctrine  (DOTD),  Training  Group,  and  instruc¬ 
tors  from  the  Ml  tank  commanders  course,  19K  BNCOC.  In  conjunction  with 
ARI  and  the  supporting  contractor,  the  Steering  Committee's  final  selec¬ 
tion  of  tasks  to  be  included  was  based  on  current  training  and  performance 
deficiencies,  and  the  feasibility  of  using  computer  based  instruction 
(CBI)  for  training  and  performing  each  task  (Knerr,  Sticha,  Ramsberger, 
Harris,  and  Tkacz,  1984).  The  following  tasks  were  selected  for  courseware 
development : 

1.  Identify  natural  terrain  features  and  determine  elevation. 

2.  Orient  a  map  to  ground  by  map-terrain  association. 

3.  Determine  location  on  the  ground  by  terrain  association. 

4.  Locate  an  unknown  point  on  a  map  or  on  the  ground  by  intersection 
or  resection. 

5.  Analyze  terrain  using  the  five  military  aspects  of  terrain. 

The  tasks  selected  are  expected  to  provide  the  most  important  skills 
required  for  tactical  land  navigation.  In  their  earlier  study,  Findlay 
et  al.  (1957)  identified  contour  visualization  and  direction  estimation  as 
the  most  critical  subset  of  tasks  required  for  daylight  land  navigation. 
The  first  three  tasks  selected  stress  map-terrain  association  and  directly 
focus  on  the  contour  visualization  problem.  The  fourth  task,  intersection 
and  resection,  addresses  direction  estimation.  The  final  task,  terrain 
analysis,  requires  the  application  of  these  basic  skills  in  a  tactical 
setting.  Before  describing  the  courseware  developed  for  training  these 
component  skills,  a  brief  discussion  of  each  task  and  topographical  map 
"reading"  requirements  is  presented. 

The  heart  of  the  land  navigation  problem  is  the  difficulty  of  deci¬ 
phering  the  information  that  is  encoded  on  topographic  maps.  A  topo¬ 
graphic  map  is  a  depiction  of  the  natural  and  man-made  features,  from  a 
small  region  of  the  earth's  surface,  that  includes  their  relative  eleva¬ 
tion  and  location.  As  McGrath  (1977)  has  documented,  the  navigator's 


primary  task  requires  the  inferential  skills  of  map  interpretation,  rather 
than  merely  map  reading.  The  cartographer's  rules  for  exclusion,  inclusion 
and  representation  of  both  natural  and  man-made  features  cannot  result  in 
a  map  that  is  the  same  as  the  earth's  surface.  As  noted  by  Rogers  and 
Cross  (1979)  a  map  is  a  stylized,  simplified,  generalized  and  codified 
presentation  of  a  selected  sample  of  some  cf  the  characteristics  of  the 
earth's  surface. 

Map-Terrain  Association.  Compounding  the  navigator's  task  is  the  re¬ 
quirement  for  contour  line  interpretation  to  visualize  the  actual  shape 
and  height  of  the  landforms  depicted,  the  terrain's  relief,  as  depicted  on 
a  topographic  map  (see  Figure  1).  The  contour  lines  connect  the  points  on 
the  map's  surface  that  have  the  same  elevation.  Terrain  relief  is  not 
only  the.  most  pervasive  and  unique  of  all  topographic  features,  it  also 
carries  great  tactical  significance.  Yet  empirical  tests  of  the  con¬ 
tour-interpretation  skills  of  experienced  military  personnel  (i.e.,  Farrel 
and  Potash,  1979;  Rogers  and  Cross,  1979)  frequently  disclose  performance 
levels  only  slightly  greater  than  chance,  and  for  many  "navigators”  a  com¬ 
plete  inability  to  visualize  the  lay- of- the- land  as  depicted  by  their 
map's  contour  lines. 

Direction  Estimation.  More  precise  skills  for  determining  one’s  posi¬ 
tion  or  the  location  of  an  unknown  point  are  resection  and  intersection, 
respectively.  The  location  of  an  unknown  point,  or  terrain  feature,  that 
is  not  depicted  on  the  map,  can  be  determined  by  plotting  onto  a  topo¬ 
graphical  map  the  azimuths  taken  from  two  or  more  locations  to  that  point. 
The  intersection  of  these  azimuths  pinpoints,  or  fixes,  the  feature's 
location.  To  determine  one's  own  position,  the  resection  of  azimuths 
plotted  from  two  or  more  features  depicted  on  the  map  identifies  one's 
current  location.  Either  compass  readings  or  line-of-sight,  straight-edge 
readings  may  be  used  for  plotting  an  azimuth.  But  given  the  unreliability 
of  compass  bearings  in  an  Armor  environment,  the  current  courseware  ad¬ 
dressed  the  straight-edge  method.  Again  the  training  and  performance  de¬ 
ficiencies  associated  with  intersection  and  resection  are  considerable. 

For  example,  Dewey  and  O'Hanlan  (1986)  in  a  resection  exercise  involving 
12  soldiers  and  3  different  locations  reported  an  average  error  of  682  me¬ 
ters,  compared  with  a  military  task  standard  of  +  100  meters. 

Terrain  Analysis.  The  final  task  included  in  the  land  navigation 
courseware,  terrain  analysis,  requires  the  application  of  map  reading 
skills  to  a  tactical  assessment  of  the  terrain.  An  analysis  of  the 
terrain  requires  the  consideration  of  tactical  concerns  such  as  cover,  con¬ 
cealment,  observation,  and  avenues  of  approach.  Ideally,  this  analysis  is 
conducted  while  performing  an  on-site  reconnaissance  of  the  area,  but  the 
presence  of  enemy  forces  may  preclude  direct  observation  of  the  terrain. 

In  this  latter  case,  map  reconnaissance,  the  terrain  analysis  requires  a 
high  level  of  contour  visualization. 

Skill  Generalization.  A  final  consideration  in  the  development  of 
land  navigation  training  is  the  problem  of  skill  generalization.  The 
Armor  force  in  particular  must  be  prepared  to  carry  out  rapid,  coordinated 
land  maneuvers  in  many  different  geographic  settings.  Yet  conventional 
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land  navigation  training,  and  especially  the  field  training,  Is  Inevitably 
limited  to  the  terrain  features  and  landforms  Indigenous  to  the  training 
locale. 


Training  Technologies 

As  part  cf  TTFA  ?  mission  to  implement  innovative  training  technolo¬ 
gies  into  Army  training,  the  land  navigation  courseware  was  designed  and 
developed  as  computer-based  instruction.  The  positive  instructional  char¬ 
acteristics  associated  with  CBI  such  as  reliability,  standardization,  cost 
effectiveness,  interactivity  and  immediate  feedback  have  been  well  articu¬ 
lated  (e.g.,  Ellis,  1986;  Heinich,  1985;  Orlansky  and  String,  1981).  A 
more  innovative  technology  included  in  the  courseware  design  was  the  use 
of  videodisc  imagery  for  depicting  live-action  modeling  of  task  perform¬ 
ance  and  dynamic  real-world  terrain  scenes.  Interactive  video  provides  a 
number  of  unique  features  such  as  fidelity  of  visual  and  audio  training 
materials,  random  and  rapid  access,  multiple  speeds  and  directions,  and 
the  potential  for  interactive  and  individualized  instruction.  A  more 
complete  discussion  of  the  merits  of  interactive  video,  beyond  the  current 
scope,  is  available  (Hannafin,  1985;  Reigeluth  5  Garfield,  1984). 

With  respect  to  the  current  TTFA  implementation,  instructional  experts 
suggest  that  interactive  video  may  be  particularly  effective  in  applied 
training  settings  (Woolley,  1982)  and  in  solving  troublesome  training 
issues  not  resolved  by  conventional  training  methods  (Thorkildson,  1983). 
In  summary,  an  interactive,  multimedia  training  package  was  developed  by 
interfacing  the  computer-based  instructional  materials,  text  and  graphics, 
with  supporting  videodisc-based  materials,  both  video  and  audio. 

Design  of  the  Courseware 

The  courseware  materials  and  instructional  methods  were  designed  and 
tailored  tc  provide  innovative  solutions  to  the  terrain  visualization  and 
contour  interpretation  problems  discussed  earlier.  In  particular  the 
transition  from  the  two-dimensional  surface  of  the  topographic  map  to  the 
three-dimensional  world  of  natural  terrain  features  and  landforms  repre¬ 
sents  a  demanding  and  nonintuitive  set  of  spatial  and  cognitive  trans¬ 
formations.  For  example,  the  inclining  terrain  associated  with  spurs  is 
portrayed  by  declining  or  U-shaped  contour  line  patterns,  and  conversely 
the  lower  ground  associated  with  draws  is  depicted  by  inclining  or  in¬ 
verted  U-shaped  contour  patterns  (See  Figure  1). 

To  facilitate  these  cognitive  transformations,  a  landform  model  was 
developed  for  this  courseware  that  included  various  terrain  features  such 
as  hills,  ridgelines,  draws  and  spurs.  This  three-dimensional  terrain 
model  was  then  annotated  with  the  appropriate  contour  patterns.  Video 
sequences  of  the  model  were  developed  that  slowly  shifted  and  reshli'ted 
from  the  overhead  or  "birds-feye"  two-dimensional  view  associated  with  map 
portrayal  to  the  horizontal,  linear  perspective  associated  with  a  direct 
view  of  the  terrain.  It  was  anticipated  that  by  being  able  to  shift  or 
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would  be  provided  unlimited  practice  at  making  the  required  perceptual 
t  ran  sformatlons. 

Special  effort  was  also  taken  to  provide  soldiers  in  a  classroom  set¬ 
ting  a  more  dynamic  view  of  real-world  terrain.  Capitalizing  on  the  rapid 
access  time  of  videodisc,  photographs  of  the  terrain  were  taken  that 
allowed  students  to  pan  across  a  360-degree  view  of  the  terrain  and  to 
zoom  in  for  close-up  views  of  this  terrain  in  any  direction. 

Finally,  the  use  of  videodisc  terrain  scenes  not  only  provided  sol¬ 
diers  high-fidelity  landscape  views,  but  a  relatively  low-cost  solution  to 
the  problem  of  non-generalized  land  navigation  training  and  skills.  The 
courseware  included  multiple  terrain  scenes  from  the  Southwest's  Mojave 
desert,  the  hilly  regions  of  Northern  Virginia,  and  the  on-site  terrain  of 
Fort  Knox,  Kentucky.  Soldiers  could  therefore  be  trained  and  tested  on  a 
cross-sample  of  very  different  geographic  regions  which  were  all  supported 
by  either  standard  1:50,000  military  maps  or  U.S.  Geological  Survey  topo¬ 
graphic  maps.  Additicnal  terrain  scenes  could  be  added  to  the  courseware 
content  to  prepare  soldiers  for  more  precise,  mission-specific,  regions. 


Objectives 

The  basic  research  question  was  to  determine  the  training  effective¬ 
ness  of  the  computer  and  videodisc-based  lard  navigation  courseware.  It 
was  hypothesized  that  the  courseware  would  significantly  increase  sol¬ 
diers'  posttest  scores,  above  their  pretest  scores,  for  each  of  the  five 
land  navigation  tasks  addressed  by  this  courseware.  Two  control  condi¬ 
tions  were  included  to  counter  the  alternative  hypotheses  that  posttest 
increases  may  have  been  caused  by  soldiers'  increased  familiarity  with  the 
pretest  items  and  the  courseware  delivery  system,  due  to  each  participants 
approximately  three  hours  of  testing  and  training.  Additional  research 
issues  were  to  evaluate  the  courseware's  training  efficiency,  in  terms  of 
time  required  to  complete  the  training,  and  the  ease-of-use  soldiers  expe¬ 
rienced  with  the  courseware. 


METHOD 


Part icipants 

Participants  for  this  research  were  E3-E5  personnel  stationed  at  Fort 
Knox  with  either  an  Armor  (MOS  19K)  or  a  Calvary  (MOS  1 9 D)  military  back¬ 
ground.  They  were  selected  as  a  sample  representative  of  the  soldiers 
that  might  receive  this  courseware  in  the  formal  program  of  instruction 
(POI)  for  the  Ml  tank  commanders,  19K  BNCOC.  Six  participants  were  in¬ 
clude©  in  a  pilot  test  of  the  land  navigation  courseware  and  experimental 
procedures.  Fourty-four  soldiers,  the  land  navigation  group,  participated 
in  the  formal  evaluation.  An  additional  39  soldiers,  the  remedial  group, 
served  as  a  control  group  for  this  evaluation.  Participants  from  both  of 
these  groups  averaged  just  over  3  years  of  military  service,  with  a  range 
of  1  to  B  years.  They  also  averaged  23  years  of  age,  w.'th  a  19  to  30  year 


6 


range.  Almost  all  (93%)  participants  had  completed  high  school,  and  nearly 
one  out  of  four  had  completed  some  college  courses.  Most  of  them  reported 
to  have  little  or  no  computer  experience.  In  general,  both  samples  ap¬ 
peared  representative  of  19K  BNCOC  attendees.  Due  to  the  formative  as¬ 
pects  of  this  evaluation  and  the  limited  number  of  student  work  stations 
available,  it  was  decided  not  to  disrupt  the  BNCOC  course  cycle  by  using 
students  currently  enrolled  in  19K  BNCOC. 


Apparatus 

The  CB1  for  this  evaluation  was  supported  by  a  MicroTICCIT  (System  II) 
host  computer  networked  to  the  student  work  stations.  Each  workstation 
included  the  following  hardware: 

o  IBM  PC  256K  bytes  RAM 
o  12”  Sony  color  monitor 
o  Sony  videodisc  player  1000  or  1000A 
o  Light  pen  and  stereophonic  headset 


All  courseware  was  developed  using  the  MicroTICCIT' s  ADAPT,  level  three, 
authoring  language.  Soldiers  were  randomly  assigned  to  one  of  the  three 
student  workstations  available  for  this  evaluation.  In  addition,  stu¬ 
dents  at  each  station  were  provided  a  set  of  paper  copy  topographic  maps, 
and  a  stereophonic  headset  to  ensure  a  direct  link  with  the  auditory  por¬ 
tions  of  the  instruction  and  to  reduce  the  probability  of  students  being 
distracted  by  any  ambient  noise. 


Design  of  the  Evaluation 

The  independent  variable  in  this  evaluation  was  the  land  navigation 
training  as  presented  by  the  computer  and  videodisc-based  courseware.  The 
dependent  measure  was  the  overall  gain  score  (posttest  minus  pretest)  for 
each  of  the  five  land  navigation  tasks.  T-tests  were  performed  for  both 
paired  and  independent  samples  as  appropriate. 

There  were  two  control  conditions  included  to  account  for  the  alterna¬ 
tive  explanations  that  significant  increases  on  posttest  measures  were  due 
to  soldiers'  increased  familiarity,  gained  fluring  the  course  of  their 
training  session,  with  the  test  items  and  tne  computer  delivery  system. 

For  each  control  condition  participants  completed  pre-  and  posttests  that 
were  unrelated  to  the  training  they  received.  For  the  first  control,  par¬ 
ticipants  in  the  land  navigation  group  completed  pre-  and  posttests  on  two 
remedial  tasks — Using  Visual  Signals  and  Establishing  Tank  Positions — 
unrelated  to  their  land  navigation  courseware.  In  the  second  control 
condition,  soldiers  from  the  remedial  group  completed  pre-  and  posttests 
on  a  land  navigation  task — Identify  Terrain  Features — before  and  after 
their  training  and  tests  on  the  remedial  tasks.  The  training  and  testing 
sequence  for  both  the  land  navigation  and  remedial  groups  is  provided  in 
Table  1.  Control  conditions,  in  which  pre-  and  posttests  were  unrelated 
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to  the  training  received,  are  also  indicated.  The  training  and  tests  for 
these  two  blocks  of  courseware,  land  navigation  and  remedial,  were  Inde¬ 
pendent  with  no  overlapping  content.  But  the  tests  and  training  for  each 
block  of  courseware  were  presented  by  the  same  computer  delivery  system. 


Table  1 

Training  and  Testing  Sequence  for  Land  Navigation  and  Remedial  Groups 


Sequence 


Group 

Pretest3 

Pretest 

Training 

Posttest 

Posttest3 

Land  Nav 

Remedial 

Land  Nav 

Land  Nav 

Land  Nav 

Remedial 

Remedial 

Land  Nav 

Remedial 

Remedial 

Remedial 

Land  Nav 

indicates 
to  training 

control  conditions  in  which  pre-  and 
received. 

posttests  were 

unrelated 

Instruments 

As  noted  above  the  primary  instrument  Involved  in  this  evaluation  was 
the  MIcroTICCIT  host  computer  and  supporting  work  stations.  All  training 
courseware  and  all  pre-  and  posttest  measures  for  both  groups  were  on-line 
products  supplied  directly  by  the  MicroTICCIT  system. 

In  addition  a  number  of  off-line  Instruments  were  also  administered 
including  a  biographical  questionnaire,  a  courseware  evaluation  question¬ 
naire,  and  a  student  record  sheet.  The  biographical  questionnaire  in¬ 
cluded  a  number  of  items  concerning  the  participant's  general  military  and 
educational  background,  computer  experience  end  self-ratings  for  selected 
abilities  (e.g.,  sense  of  direction).  The  courseware  evaluation  question¬ 
naire  contained  a  small  number  of  open  and  closed-ended  Items  that  asked 
the  participants  to  rate  the  overall  training  effectiveness  of  the  course¬ 
ware  with  respect  to  conventional  land  navigation  training  and  to  provide 
any  recommendations  they  might  have  for  improving  the  courseware.  Finally 
the  student  record  sheet  which  was  maintained  by  the  classroom  proctors 
provided  a  data  sheet  for  recording  summary  scores,  rime  requirements  for 
training  and  testing,  and  a  log  for  recording  system  malfunctions,  student 
problems  and  proctor  Interventions.  Complete  copies  of  each  of  these 
off-line  instruments  are  provided  In  Appendix  A. 


Procedure 

Soldiers  were  trained  and  tested  Individually  as  they  worked  at  their 
assigned  student  work  station.  Prior  to  the  training  session  each 
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participant  received  a  brief  on-line  introduction  and  orientation  to  the 
MicroTICCIT  system.  This  introduction  explained  the  organizational  struc¬ 
ture  of  the  courseware  and  the  supporting  menu  and  prompt  messages.  In 
addition,  this  introduction  familiarized  students  with  the  use  of  the 
light  pen  and  the  availability  of  an  Advisor  function  to  assist  them 
through  the  courseware.  All  student  inputs,  requests  and  answers  were 
entered  by  using  a  light  pen.  After  each  participant  completed  this  ori¬ 
entation,  pretests  were  administered  for  each  task.  Upon  completion  of 
all  pretests,  participants  from  the  land  navigation  group  began  their 
training  on  the  land  navigation  courseware,  and  participants  from  the  re¬ 
medial  group,  on  the  remedial  courseware. 

The  training  modules  for  each  of  the  five  land  navigation  tasks  were 
developed  with  the  same  structure  that,  with  only  minor  exceptions,  in¬ 
cluded  the  objective,  the  instruction,  a  review,  a  help  section,  practice, 
and  the  pre-posttests  (see  Figure  2).  While  all  participants  were  di¬ 
rected  to  complete  the  initial  objective  and  instruction  components,  their 
training  curriculum  thereafter  was  determined  by  their  self-selections  and 
computer-based  prompts.  Instructional  prompts  based  on  the  soldier's 
response  and  error  patterns  were  provided  by  the  on-line  Advisor.  For 
example,  a  student  making  an  error  on  practice  problems  might  automatic¬ 
ally  be  referred  by  the  Advisor  to  a  related  help  component.  After  suc¬ 
cessfully  answering  a  pre-set  number  of  practice  problems  the  student 
would  be  informed  by  the  Advisor  that  he  was  probably  ready  to  take  the 
appropriate  test. 

The  posttest  for  each  task  was  administered  immediately  after  the 
training  for  that  task  was  completed.  Participants  then  proceeded  through 
training  on  the  remaining  tasks,  maintaining  the  same  train-test  sequence. 
All  participants  were  allowed  to  take  short  breaks  at  their  discretion, 
but  they  were  instructed  not  to  discuss  the  training  and  test  material 
until  after  they  had  completed  all  courseware  and  post-test  measures. 

After  finishing  all  training  and  related  posttests,  participants  were 
administered  posttest(s)  on  the  control  task(s)  for  which  they  had  re¬ 
ceived  no  training.  All  participants  were  then  asked  to  complete  a  very 
brief  courseware  evaluation  questionnaire,  and  the  classroom  proctors 
recorded  any  extended  comments  or  recommendations  they  made  about  improv¬ 
ing  the  courseware  and  training  procedures. 

Each  soldier's  session  lasted  approximately  three  hours  depending  upon 
the  participant's  pace  in  completing  the  training  and  testing  require¬ 
ments.  Throughout  the  entire  session,  classroom  proctors  were  present  to 
assist  students  with  any  problems,  input  proctor  functions  for  accessing 
the  tests  which  were  not  directly  accessible  to  students,  and  record  both 
test  scores  and  participants'  comments. 


Evaluation  Measures 

All  criterion  measures,  the  pre-  and  posttest  scores,  were  generated 
from  on-line  courseware  modules.  Tests  for  each  of  the  five  land 
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navigation  and  two  remedial  tasks  contained  close-ended  multiple  choice 
items.  Students  indicated  their  answers  by  selecting  with  the  light  pen 
their  choice  from  among  those  response  alternatives  appearing  on  the  moni¬ 
tor.  Number  of  test  items  and  choices  per  item  varied  as  a  function  of 
courseware  content  and  design.  A  brief  description  of  each  test  is  pro¬ 
vided  below. 

Identify  natural  terrain  features.  This  test  consisted  of  18  items  of 
three  different  types.  For  six  of  the  items  students  were  presented  a 
video  image  of  a  topographic  map  and  a  list  of  terrain  feature  labels 
(e.g.,  hill,  spur,  valley).  Soldiers  were  required  to  select  the  feature 
name  associated  with  the  contour  pattern  that  was  highlighted  by  a  graphic 
overlay.  For  six  other  items  students  were  again  provided  the  video  image 
of  a  topographic  map  and  a  list  of  feature  names,  but  now  asked  to  mark 
the  contour  pattern  on  the  map  for  a  designated  feature  name.  For  the 
final  six  test  items  students  were  provided  a  video  image  of  a  topographic 
map  and  asked  to  determine  the  elevation  of  the  terrain  feature  designated 
by  a  blinking  arrow. 

Determine  a  location  on  the  ground  by  map  terrain  association.  This 
test  consisted  of  six  items.  Students  were  allowed  to  freely  pan  360  de¬ 
grees  across  videodisc  images  of  the  terrain,  t  nd  then  required  to  associ¬ 
ate  these  features  with  those  depicted  around  three  different  possible 
locations  designated  on  their  maps.  Answers  were  input  by  selecting  from 
the  monitor  one  of  the  three  locations  (a,  b,  or  c)  similarly  labelled  on 
their  paper  map. 

Orient  a  map  to  the  ground  by  map  terrain  association.  This  six  item 
test  also  provided  students  the  capability  of  panning  360  degrees  across 
videodisc  images  of  the  terrain  and  required  them  to  associate  these  ter¬ 
rain  features  with  those  depicted  on  their  paper  map.  Since  the  computer 
system  could  not  detect  the  orientation  of  their  paper  map,  students  were 
asked  to  select  any  key  terrain  feature  appearing  on  the  video  monitor  and 
indicate  its  cardinal  direction  as  portrayed  on  the  corresponding  paper 
map.  Twenty-two  directional  choices  (N,  NNE,  ENE,  E,  etc.)  were  depicted 
on  a  circular  response  panel  graphically  portrayed  on  the  work  station 
monitor.  The  standard  for  "correct"  orientation  was  set  at  +  30  degrees. 

Locate  an  unknown  point  on  the  ground  by  Intersection  or  resection. 
This  10  item  test  required  students  to  designate  the  correct  order  of  the 
procedures  for  intersection  and  resection.  Soldiers  were  given  a  video 
image  of  a  topographical  map  and  an  incomplete  intersection  or  resection 
task  (e.g.,  only  one  azimuth  drawn  from  a  key  terrain  feature  to  the  un¬ 
known  point)  and  asked  to  select  the  next  procedural  step  for  completing 
the  task. 

Analyze  terrain  using  the  five  military  aspects  of  terrain.  This  test 
consisted  of  20  items  of  three  difference  types.  For  the  first  part  stu¬ 
dents  were  required  to  list  the  five  aspects  of  OCOKA  (observation,  cover 
and  concealment,  obtacles,  key  terrain,  and  avenues  of  approach)  in  the 
correct  order.  In  the  second  part,  students  were  required  to  match  each 
of  these  aspects  of  terrain  with  their  correct  meaning  or  definition.  For 
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the  third  and  final  part,  students  were  presented  with  color  graphics  of 
battlefield  terrain  scenes  and  asked  to  rate  the  CCOKA  related  advantages 
or  disadvantages  of  a  given  aspect  of  th~  terrain. 

Pre-  and  posttest  scores  were  obtained  by  assigning  one  point  to  each 
item  correctly  answered.  To  ensure  equivalent  measures,  identical  items 
were  used  for  both  the  pre-  and  posttests.  For  a  more  complete  descrip¬ 
tion  of  the  test  items  and  courseware  description  the  reader  is  referred 
to  Elder,  Harris,  Sticha,  Stein,  Knerr,  and  Tkacz  ( 1985 ) . 


RESULTS 


Tasks  Not  Trained 

As  previously  discussed  the  research  design  for  this  evaluation  was  a 
pretest-posttest  design  in  which  the  effects  of  training  are  based  on  im¬ 
provements  or  gains  in  students'  posttest  scores  over  and  above  their  pre¬ 
test  scores.  Higher  posttest  scores,  however,  may  have  been  due  to 
students  increased  familiarity  with  the  computer-based  training  delivery 
system  and  the  test  items  and  formats,  rather  than  the  training  itself. 

To  control  for  these  potential  effects  each  parcicipant  from  the  land 
navigation  group  completed  on-line  tests  for  two  non-trained  tasks  before 
and  after  working  with  the  land  navigation  courseware. 

A  summary  of  the  data  for  these  tasks  not  trained — Using  Visual  Signal 
and  Establishing  Tank  Positions — is  presented  in  the  upper  portion  of 
Table  2.  The  average  pre-  and  posttest  scores  for  the  Signals  task  were 
li.02  and  ii.45  and  for  the  Position  Task,  6.32  and  6.63.  Paired  compari¬ 
son  t-tests  found  no  significant  difference  between  the  pre-  and  post 
scores  for  the  Signals  task,  _t  (43)  -  1.61,  j)  >  .12,  or  the  Positions 
task,  _t  (42)  =  -.96,  >  .30. 

More  direct  evidence  for  ruling  out  the  effects  of  computer-experience 
on  the  land  navigation  posttests  per  se  is  provided  by  data  from  the  reme¬ 
dial  group  also  included  in  Table  2.  The  remedial  group  was  tested,  but 
not  trained,  on  one  of  the  land  navigation  tasks — Identify  Terrain  Fea¬ 
tures.  The  remedial  group's  pretest  average  of  10.78  and  posttest  average 
of  li.41  for  this  task  were  not  significantly  different,  _t  (38)  =  1.48,  j) 

>  .14.  These  results  for  tasks  not  trained  indicate  that  familiarity  with 
the  computer  delivery  system  and  the  test  items  did  not  account  for  any 
significant  improvements  on  the  land  navigation  posttests. 


Tasks  Not  Evaluated 

Training  courseware  for  two  of  the  land  navigation  tasks — Determine  a 
Location  and  Orient  a  Map  by  Terrain  Association — were  not  formally 
evaluated  due  to  inadequate  courseware  design.  The  design  problem  unique 
to  both  of  these  tasks  was  how  to  best  display  a  360-degree  horizontal 
view  of  the  terrain  on  a  single  workstation  monitor,  a  TV  screen.  The 
solution  attempted  in  the  current  courseware  design  was  to  simulate  a 
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Table  2 


Summary  of  Results  for  Tasks  Trained  and  Tasks  Not  Trained 


Number  of 

Number  of 

Pretest 

Posttest 

Gain 

Tasks 

Soldiers 

Items 

Average 

Average 

Average 

Not  Trained 


Signals 

44 

16 

11.02 

11.45 

.43 

Positions 

43 

12 

6.32 

6.63 

.31 

Features3 

39 

18 

10.78 

11.41 

.63 

Trained 

Features 

42 

18 

11.12 

13.07 

1.95 

Location 

44 

10 

2.86 

7.48 

4.62 

Terrain 

44 

20 

10.00 

16.34 

6.34 

Note :  Task  titles  refer  in  order  to  the  following  tasks:  Communicate 
Using  Visual  Signals,  Establish  Tank  Positions,  Identify  Terrain  Features, 
Determine  location  by  Intersection  or  Resection,  and  Analyze  the  Five 
Military  Aspects  of  Terrain. 

indicates  data  from  the  remedial  group. 


visual  pan  across  the  terrain  by  rapidly  accessing  from  the  videodisc  120 
sequentially  stored  video  frames  with  each  frame  incrementing  the  pan  by 
3  degrees.  While  the  pan  simulation  appeared  realistic,  soldiers  were 
unable  to  integrate  this  linear  set  of  frames  into  a  circular  (360- 
degree),  composite  view  of  the  surrounding  terrain.  Soldiers  were  espe¬ 
cially  distressed  by  their  inability  to  infer  the  angular  displacement 
between  terrain  features  that  could  not  be  simultaneously  viewed  on  the 
monitor. 

Students  were  having  such  difficulty  with  these  tasks  that  Orient  a 
Map  was  dropped  from  the  evaluation  during  the  pilot  phase  prior  to  the 
evaluation,  and  Determine  a  Location  was  eliminated  during  the  first  half 
of  the  four  week  evaluation  cycle.  Participants'  frustration  with  these 
tasks  was  evident  both  In  their  posttest  subjective  evaluation  of  the 
courseware,  and  their  posttest  scores  that  were  lower  than  their 
pretests.  The  experimenters  decided  that  participants'  frustration  with 
these  tasks  might  reduce  their  motivation  on  the  other  tasks  and  that  it 
was  unreasonable  to  force  the  students  to  work  on  these  training  modules. 
A  more  complete  discussion  of  these  tasks  and  proposed  modifications  to 
their  courseware  design  are  presented  in  the  following  section. 
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Tasks  Trained  and  Evaluated 


Training  Effectiveness.  Summary  data  for  the  three  tasks  formally 
evaluated — Identify  Terrain  Features,  Intersection/Resection  and  Terrain 
Analysis — are  presented  in  the  lower  portion  of  Table  2.  The  average  pre- 
and  posttest  scores,  respectively  were  11.12  and  13.07  for  Identify  Fea¬ 
tures,  2.86  and  7.48  for  Intersection/Resection,  and  finally  10.00  and 
16.34  for  Analyze  Terrain.  Paired  comparison  t-tests  for  each  of  these 
tasks  found  significant  improvements  in  posttest  performance.  The  t-test 
values  and  significance  levels  were  4.44  and  .003  for  identify  Features, 
13.22  and  .000  for  Intersection  and  Resection,  11.98  and  .000  for  Analyze 
Terrain. 

The  courseware's  training  effectiveness  for  the  latter  two  tasks — In¬ 
tersection/Resection  and  Analyze  Terrain — is  apparent  and  would  override 
any  alternative  explanation  based  on  soldier  familiarity  with  test  items 
and  the  computer  delivery  system.  For  the  other  task — Identify  Terrain 
Features — the  paired  comparison  is  significant,  but  not  conclusive.  For- 
tunately,  the  remedial  group  serves  as  a  direct  control  for  this  task 
since  they  also  completed  its  pre-  and  posttests  without  the  benefit  of 
feature  identification  training,  and  on  the  average  four  hours  of  remedial 
training  on  the  same  computer  delivery  system.  A  between  group,  land 
navigation  versus  remedial,  t-test  comparison  of  their  gain  scores  for 
Identify  Terrain  Features  was  significant,  _t  (79)  =  2.08,  £  =  .041.  This 
finding  of  a  significantly  greater  gain  for  the  land  navigation  group 
effectively  rules  out  alternative  explanations  of  increased  familiarity 
with  test  formats  and/or  the  computer  delivery  system. 

A  more  practical  index  of  the  courseware's  training  effectiveness  is 
the  relation  of  these  pre-  and  posttest  values  to  the  P01  standards  for 
passing  the  19K  BNCOC  course.  Table  3  summarizes  this  comparison.  In 
terms  of  "class"  averages,  participants  began  their  training  for  each  of 
these  tasks  at  a  failing  or  “No  Go"  level  and  completed  their  training  at 
a  passing  or  "Go"  rate.  Their  relatively  low  pretest  scores  on  lnte^sec- 
tion/Resection  and  Terrain  Analysis  set  the  stage  for  remarkably  large 
posfctest  gains.  Similarly  their  relatively  high  pretest  performance  on 
Identify  Terrain  Features,  may  at  least  partially  account  for  this  train¬ 
ing's  more  modest  posttest  improvement. 


Table  3 

Percentage  of  Test  Items  Correctly  Answered:  Comparison  of  Course  Stan¬ 
dard  for  Passing  with  Pretest  and  Posttest  Results 


Task 

Standard 

Pretest 

Posttest 

Gain 

Features 

70 

62 

73 

11 

Location 

70 

29 

75 

46 

Terrain 

80 

50 

82 

32 
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Finally,  soldiers  themselves  reported  that  the  computer-based  course¬ 
ware  was  more  effective  than  conventional  training.  On  a  9-point  rating 
scale  their  average  effectiveness  rating  for  this  C81  was  6.95  (7.0  = 
"Effective”)  as  compared  to  6.27  for  conventional  land  navigation  train¬ 
ing,  _t  (40)  =  2.52,  g  =  .016. 

Training  Efficiency.  Training  efficiency  is  also  a  critical  consid¬ 
eration  in  the  evaluation  of  an  applied  training  methodology.  The  data  in 
Table  4  compare  the  training  efficiency,  in  terms  of  time  required  to 
complete  training  and  testing,  of  this  computer  based  instruction  (CBl) 
with  the  efficiency  of  the  conventional  program  of  Instruction  (POl).  Data 
are  presented  for  each  of  three  tasks  formally  evaluated,  and  expressed  in 
terms  of  minutes  required  for  completion.  CBl  data  were  obtained  directly 
from  the  computer  delivery  system  and  the  POI  data  were  extracted  from  the 
training  schedules  for  the  19K  BNCOC  course  cycle.  On  the  average  two 
hours  were  required  to  complete  the  CBl  training  and  testing  for  all  three 
tasks  compared  with  five  hours  for  the  conventional  POl. 

Much  of  the  efficiency,  as  well  as  the  effectiveness,  of  CBl  is  due  to 
individualized  instruction.  The  land  navlgatien  courseware  was  designed 
to  provide  a  training  package  that  readily  adapted  to  each  student's  entry 
level  and  ability.  The  Advisor  function  recommended  to  students  when  they 
were  ready  to  take  the  test  based  on  their  performance  with  practice  prob¬ 
lems,  or  branched  them  back  to  the  help  and  review  components.  The  wide 
range  of  times  required  for  students  to  complete  their  training  and  test¬ 
ing,  less  than  1  hour  at  a  minimum  to  nearly  3.4  hours  at  the  maximum,  at 
least  partially  accounts  for  the  courseware's  training  efficiency  compared 
to  the  POl's  conventional,  lock-step  schedule. 

Table  4 

Average  Completion  Times  for  CBl  Training  and  Testing  Versus  POl 


Task 

CBl 

Range 

CBl 

Total3 

POl 

Total8 

Gain 

Identify 

26-110 

65 

120 

55 

Locate 

15-38 

25 

120 

95 

Analyze 

13-55 

30 

60 

30 

71-203 

120 

200 

180 

aTotal  indicates  the  time  required  to  complete  training  and  posttests, 
pretest  times  are  not  included  because  they  re  not  performed  in  conven¬ 
tional  training  for  BNCOC.  All  completion  il  s  are  reported  in  minutes. 


15 


Ease  of  Training.  Another  major  consideration  in  the  evaluation  of  an 
instructional  system  is  the  soldiers'  ease-of-use  in  adapting  to  the 
courseware's  content,  procedures,  and  requirements.  Perhaps  the  most 
objective  evidence  of  this  courseware's  usability  is  the  reduced  training 
time  when  combined  with  significant  posttest  improvements.  But  the  par¬ 
ticipants'  subjective  evaluations  of  the  courseware,  from  the  Posttest 
Questionnaire,  were  also  quite  favorable.  Frequently  these  comments  were 
"easy  to  use",  "explanations  were  clear  and  simple"  and  "better  than  a 
book".  More  telling  comments  with  respect  to  the  courseware's  adaptability 
and  student  diversity  ranged  from  "fast  and  to  the  point",  "feedback  was 
quick  and  direct"  to  "able  to  go  back  and  review",  "patient". 


DISCUSSION 


Effective  Courseware 

Results  from  the  land  navigation  courseware  modules  tested  during  this 
evaluation  demonstrate  the  potential  for  using  innovative  training  tech¬ 
nologies  in  an  applied  military  setting.  For  each  of  the  three  tasks 
tested — Identify  Terrain  Features,  Intersection  and  Resection,  and  Analyze 
Terrain — the  courseware  resulted  In  significant  improvements  In  posttest 
performance.  More  relevant  to  the  training  goals,  these  were  Improvements 
that  raised  the  average  score  for  each  of  these  tasks  from  "No  Co"  to 
"Co". 

Several  features  of  this  evaluation  should  be  noted  chat  may  have  con¬ 
tributed  to  an  underestimation  of  this  courseware's  training  effective¬ 
ness.  First,  the  fact  that  a  representative  sample  of  soldiers,  rather 
than  actual  soldiers  from  I9K  BNCOC,  were  used  in  this  evaluation  may  have 
reduced  the  motivation  and  attentiveness  typical  of  a  student  concerned 
with  passing  the  course.  Secondly,  although  the  courseware's  initial  tar¬ 
get  audience  is  19K  BNCOC  with  soldiers  averaging  over  3  years  of  military 
training,  more  significant  posttest  improvements  might  be  obtained  with 
entry  level  personnel  who  must  also  be  trained  on  land  navigation  skills. 
Finally,  mastery  learning  was  not  required  In  this  evaluation  and  posttest 
"Go"  rates  might  have  been  significantly  Improved  if  soldiers  receiving  a 
"No  Co"  had  beers  required  to  review  and  retnkc  the  posttest.  All  soldiers 
in  19K  BNCOC  are  given  three  posttest  opportunities  for  each  task. 

The  courseware  for  each  of  these  tasks  also  proved  more  efficient  than 
conventional  training  procedures  as  evidenced  by  a  60X  reduction  In  time 
required  to  complete  the  training.  In  addition,  the  courseware  for  each 
of  these  tasks  proved  remarkably  easy  for  soldiers  to  use.  The  reduced 
training  time,  given  training  effectiveness,  suggests  that  students  had 
little  difficulty  mastering  the  basic  Interactive  tasks  required  for  util¬ 
izing  the  courseware.  Ease-of-use  is  further  evidenced  by  the  fact  that 
students  showed  no  significant  improvement  on  the  posttests  for  tasks  that 
were  not  trained.  The  fact  that  students  scored  as  well  on  pretests  for 
tasks  not  trained  as  they  did  on  identical  posttests,  after  several  hours 
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of  interacting  with  the  computer  delivery  system,  suggests  they  quickly 
adapted  to  much  of  this  automated  training  package. 


Ineffective  Courseware 

For  the  two  other  tasks — Determine  Location  and  Orient  a  Map  by  Ter¬ 
rain  Association — developed  as  part  of  the  land  navigation  courseware, 
results  were  not  favorable. 

The  design  problem  unique  to  both  of  these  tasks  is  the  requirement  to 
maintain  orientation  of  a  360-degree  horizontal  pan  of  the  terrain  when 
presented  on  a  single  monitor  with  its  limited  f ield-of-view.  The  re¬ 
quirement  is  analogous  to  a  tank  commander’s  task  of  viewing  the  terrain 
while  looking  through  an  Isolated  vision  block  as  the  tank’s  turret  Is 
rotated.  This  formidable  requirement  in  the  tank  environment  is  com¬ 
pounded  in  the  current  courseware  design  by  the  absence  of  correlated 
vestibular  cues,  since  the  viewer  remains  stationary  and  the  terrain  ro¬ 
tates.  Each  of  the  courseware’s  360-degree  pans  of  the  terrain  were 
based  on  120  video  frames  that  incremented  the  soldier's  field  of  view  by 
3  degrees  as  he  "rotated"  to  view  the  surrounding  landscape.  Even  when 
soldiers  were  able  to  Identify,  or  associate,  two  or  more  terrain  features 
with  those  depicted  on  the  topographic  map,  they  had  great  difficulty 
inferring  the  angular  displacement  between  these  features. 

In  anticipation  of  this  problem,  the  courseware  designers  Included  an 
Icon  of  a  moving  arrow  on  the  display  that  continually  updated  the  angular 
distance  the  subject  had  rotated  from  the  initial  frame.  At  the  termina¬ 
tion  of  each  pan  (from  0  degrees  to  360  degrees)  Initiated  by  the  stu¬ 
dent,  the  arrow  rotated  and  depicted  the  angular  displacement  between  the 
student's  current  view  and  his  initial  view  of  the  terrain.  This  solution 
was  inadequate. 


Courseware  Modifications 

Is  there  a  solution  to  this  problem  given  the  hardware  configuration, 
one  display  monitor  per  student  workstation,  of  the  current  courseware? 
Other  technologies  are  available  for  providing  a  simultaneous  360-degree 
horizontal  view  of  the  terrain,  but  they  are  not  consistent  with  TTFA's 
goals  of  providing  low  cost,  self-paced  and  readily  transferable  training 
methodologies.  For  example,  ARI-Fort  Benning  has  developed  a  training 
device  called  SURNOT  (Surface  Navigation  and  Orientation  Trainer)  that 
utilizes  a  fish-eye  lens  for  photographing  and  projecting  360-degree 
perspective  fields  of  view  (Dewey  and  O'ilanlan,  1986).  SURNOT  would 
appear  to  be  an  excellent  training  device  for  these  tasks,  but  its  current 
hardware  configuration  does  not  meet  any  of  the  TTFA  goals  previously 
discussed. 

Based  on  students’  suggestions  and  the  experimenters'  observations 
several  recommendations  for  reducing  this  disorientation  while  maintaining 
the  current  delivery  system  are  being  pursued.  These  recommendations 


wrai/  *  w  *  ira  vw.  tw  wwwm,mw'^  i  ■  ,ii,?'wr  mi '.»  •-'  ,*,*  mwimw  .wvre*  w 


wj  » w  wmm  wiiii  «  w  ii  i  uw'jii  r im  i» 


m 
[-*.1 
!■  .  «J 

«v- 


'-V- 


focus  primarily  on  redesign  of  the  workstation's  control  panel  by  which 
the  student  directs  the  panning  sequences.  One  alternative  design  is  to 
replace  the  moving  arrow  graphic  with  a  generic  compass  overlay  by  which 
students  could  dl'ectly  select  angular  displacements  of  interest.  The 
compass  is  generic  in  that  it  provides  only  relative  direction  and  rela¬ 
tive  angular  displacements,  since  more  absolute  bearings  such  as  cardinal 
directions  would  circumvent  the  purpose  of  the  task.  The  configural  na¬ 
ture  of  this  graphic,  as  opposed  to  the  discrete  arrow  indications,  would 
provide  soldiers  a  frame  of  reference  for  mentally  computing  the  angular 
displacement  between  selected  features.  In  addition,  as  the  user's  control 
panel,  this  compass  would  maximize  the  random  access  capability  of  the 
videodisc  and  allow  users  to  rapidly  or  even  directly  shift  from  one  view 
of  the  terrain  to  another  without  examining  all  the  Intervening  frames  on 
the  videodisc.  While  this  exclusion  of  intervening  terrain  might  appear 
disruptive,  it  is  consistent  with  the  natural  processing  of  perceptual 
information.  When  a  viewer  rapidly  shifts  the  direction  of  gaze,  only  the 
original  and  final  views  are  perceived  in  detail.  This  and  several  addi¬ 
tional  design  modifications  for  the  courseware  In  general  are  being  devel¬ 
oped  in  a  follow-up  effort  to  this  evaluation. 


Courseware  Implementation 

Based  on  the  results  of  this  evaluation,  recommendations  for  course¬ 
ware  implementation  were  made  to  the  TTFA  management  team.  The  first  rec¬ 
ommendation  was  that  the  computer  and  videodisc- based  land  navigation 
courseware  for  three  tasks  formally  evaluated — Identify  Terrain  Features, 
Determine  a  Location  by  Intersection  or  Resection,  and  Analyze  Terrain — be 
validated  for  the  basic  POI  of  19K  BNCOC.  The  second  recommendation  was 
that  the  courseware  for  the  other  two  tasks — Orient  a  Map  and  Determine 
Location  by  Map  Terrain  Association — not  be  validated  until  courseware 
design  modifications  for  these  tasks  had  been  made  and  formally  evaluated. 
Acting  on  these  recommendations,  validatic  trials  have  been  scheduled  for 
those  tasks  recommended.  The  validation  trials,  described  below,  are  the 
final  TTFA  evaluation  prior  to  courseware  Implementation. 

For  the  validation  trials,  actual  students  from  the  19K  BNCOC  course 
are  used  as  participants  and  all  instructor  and  proctor  functions  are 
performed  by  the  1 9 K  BNCOC  course  Instructors,  rather  than  the  TTFA  and 
ARI  support  personnel  used  during  this  evaluation.  In  addition  the  vali¬ 
dation  trials  are  designed  to  provide  a  more  direct  comparison  hetween  the 
training  effectiveness  of  the  computer  and  videodisc-based  instruction 
(CBI)  with  the  conventional  training  from  the  current  program  of  instruc¬ 
tion  (POI).  Due  to  19K  BNCOC’s  limited  class  size,  12-25  students,  the 
validation  trials  will  be  conducted  over  several  course  cycles  to  provide 
a  sufficient  data  base. 

The  validation  trials  were  initiated  during  the  first  regular  course 
cycle  following  this  evaluation  and  the  results,  although  preliminary,  are 
favorable.  One  half  (6  students)  of  the  class  received  their  land  naviga¬ 
tion  training  on  the  computer  and  videodisc-based  courseware.  They  then 
completed  pcsttests  on  this  system,  and  on  the  conventional  paper  and 
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pencil  test  formats  currently  used  in  the  course.  All  of  these  students 
passed  both  the  CBI  and  POI  tests  on  their  first  trial,  with  the  exception 
of  one  student  on  one  of  the  tasks  for  both  mediums.  If  the  remaining 
validation  trials  prove  as  successful,  they  will  set  the  stage  for  imple¬ 
mentation  of  this  courseware  into  19K  BNCOC. 
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APPENDIX  A 

EVALUATION  INSTRUMENTS 

BIOGRAPHICAL  QUESTIONNAIRE 


Subject  $ 

Date 

Social  Security  $ 

Rank  E  - 

—  0  - 

1.  How  long  have  you  been  in  the  service? 

years 

months 

2.  How  old  are  you? _ years  _ _ months 

3.  What  is  your  primary  MOS/SC?  _ 

What  is  your  secondary  MOS/SC?  _ 

5.  WTiat  is  your  current  Unit? _ _ 

6.  Please  check  each  of  the  following  courses  you  have  attended: 

PLDC  _  BNCOC  _  ANCOC  _  AOC  _  AOAC  _____ 

7.  Check  each  of  the  following  statements  that  are  true  for  you. 

_  _  I  have  no  previous  experience  with  computers. 

I  frequently  play  video  games. 

_  I  own  a  home  computer. 

I  have  received  formal  training  on  how  to  use  a  computer. 

I  have  previously  participated  in  training  delivered  by  a  computer. 

I  have  a  lot  of  experience  with  computers. 

8.  What  is  your  highest  academic  level  completed?  (Check  one  below.) 

High  School  _  _  CED  _  Some  College  _  College  Graduate 

9.  How  would  you  describe  your  "handedness"  o^  preferred  hand?  (Check  one 
below. ) 

Right  handed  _  Left  handed  Both  hands 
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10.  How  would  you  rate  your,  "sense  of  direction",  your  ability  to  maintain 
your  orientation?  Please  circle  the  number  below  (1-9)  that  best  de¬ 
scribes  your  sense  of  direction. 


1  .  2 

3 

1) 

5  6 

7 

8 

9 

Extremely 

Poor 

Average 

Good 

Extremely 

Poor 

Direction 

Direction 

Di rection 

Good 

Direction 

Direction 

1 1 .  How  would  you 

rate  your 

ability  to  perform 

terrain  visualization,  that 

is 

your  ability  to  associate  terrain  and  cultural  features 

with 

those 

depicted  on  the  topographic 

map?  (Please  < 

circle  the  number 

below  that 

best  describes  your  terrain 

association  abilities.) 

1  2 

3 

5  6 

7 

8 

9 

Extremely 

Poor 

Average 

Good 

Extremely 

Poor 

Association 

Association 

Association 

Good 

Association 

Association 

12.  Would  you  say 

that  you  were 

raised  more  as 

"country"  boy 

or 

as  a  "city" 

boy?  (Please 

circle  a  number  below  that  best  describes 

your 

background 

.) 

1  2 

3. 

1) 

5  6 

7 

8 

9 

Country 

Mostly 

Half  Country 

Mostly 

City 

Boy 

Country 

Half  City 

City 

Boy 

13-  In  general  how  would  you 

rate  your  ability 

to  perform  military  land 

navigation  tasks. 

Please  circle 

the  number  below  (1-9)  that 

best 

describes 

your  ability  as  a 

navigator. 

• 

1  2 

3 

1) 

5  6 

7 

6 

9 

Extremely 

Poor 

Average 

Good 

Extremely 

Poor 

Navigator 

Navigator 

Navigator 

Good 

Navigator 

Navigator 

1*J.  How  long  has 

it  been  since 

you  last  had  the  opportunity 

or  requirement 

to 

perform  some  land 

navigation 

tasks  in  a  field  setting?  Please  check  one  of 

the 

times  below. 

Less  than  6 

months 

3- 

5  years 

- 

- 

6  months  to  a  year  core  than  5  years 


POST  TESTING  QUESTIONNAIRE 


Subject  ii  _ _  Date  _ 

We  would  like  to  ask  you  a  few  final  questions  about  how  you  would  evaluate 
this  computer  assisted  training  that  you  have  just  experienced. 

1.  In  general  what  did  you  like  best  about  this  computer-assisted  training? 
Please  briefly  write  your  answer  below. 


2.  In  general  what  did  you  least  like  about  this  computer-assisted  training? 
Please  briefly  write  your  answer  below. 


3.  Now  we  would  like  you  to  rate  the  effectiveness  of  this  computer  assisted 
training.  How  would  you  rate  the  overall  training  effectiveness  of  the 
instruction  you  have  just  completed  for  training  land  navigation  skills? 
Please  circle  the  number  below  (1-9)  that  best  indicates  your  evaluation  of 
the  instructions  training  effectiveness. 


1  2 

3 

ll  5  6 

7  8 

9 

Very 

Ineffective 

Training 

Ineffective 

Average 

Effectiveness 

Eff  ective 

Very 

Effective 

Training 

JJ,  In  comparison,  how  would  you  rate  the  training  effectiveness  of  the 
standard  or  "conventional"  instruction  provided  for  land  navigation. 

Eased  on  your  military  training  please  circle  the  number  below  (1-9)  that 
best  indicates  your  evaluation  of  the  conventional  land  navigation 
training. 


1  2  3 

JJ  5  6 

7 _ 8 

9 

Very  Ineffective 

Average 

Effective 

Very 

Ineffective 

Effectiveness 

Effective 

Training 

Training 

5.  Wliat  could  be  done  to  improve  this  training?  Please  write  suggestions 
below. 
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PROCEDURES  FOR  LAND  NAVIGATION  TRIALS 


1.  Give  students  an  orientation  to  the  day's  activities.  Cover  the  following 
points: 

a.  Explain  the  purpose  of  the  study;  this  is  to  test  computer-based  train¬ 
ing  for  BNCOC  before  using  it  in  the  eourse.  Tell  students  that  no  record  of 
their  performance  will  be  maintained  or  returned  to  their  unit.  But  they 
should  still  do  the  best  they  can  with  the  eomputer-based  training,  since  they 
may  get  some  useful  training  out  of  it  and  their  performance  will  affect  what 
they  will  see  in  BNCOC  someday.  Motivate  the  students  to  give  an  honest  ef¬ 
fort  . 


b.  Give  the  students  an  overview  of  the  day's  activities.  Explain  that 
they  will  take  a  series  of  tests  on  the  computer  followed  by  training  and 
retests.  Tell  them  to  raise  their  hands  or  call  you  before  they  start  a  block 
of  instruction  or  a  test,  and  when  they  have  problems.  Tell  them  not  tc  be 
frustrated  by  the  initial  tests,  but  to  do  the  best  they  can. 

e.  A  few  remarks  about  scoring  and  "not  doing  well".  First,  some  of  the 
problems  may  be  very  difficult  and  the  standards  for  determining  "correct"  vs 
"incorrect"  and  "Go"  vs  "No-Go"  are  in  sone  eases  too  demanding.  Don't  worry 
about  how  the  machine  "scores"  your  performance,  this  is  being  evaluated  also. 
Second,  we  are  asking  you  to  take  the  land  navigation  tests  before  you  receive 
any  land  navigation  training.  This  is  not  standard  training  procedure  but  only 
a  requirement  for  this  particular  evaluation.  So  don't  be  to  concerned  about 
scores,  just  do  the  best  you  can. 

d.  Cover  administrative  points  and  answer  student’s  questions.  Explain 
that  breaks  and  lunch  period  will  be  taken  as  needed  (within  reason). 

Emphasize  that  you  are  available  to  help  them,  but  you  will  not  tell  them  the 
answer  or  eoach  them. 

2.  Assign  student  numbers.  Training  and  posttest  number  should  always  br  100 
higher  than  pretest  number.  The  operator  should  already  have  registered 
students  under  these  numbers. 

3.  Administer  the  biographical  questionnaire  to  each  student.  Be  available  to 
help  them  complete  it. 

4.  Move  the  students  to  KicroTICCIT  work-station.  Place  each  student's  record 
sheet  at  his  work-station.  Try  to  ease  any  fear  they  have  of  the  computer. 

Show  then  how  the  light-pen  works  and  have  them  take  the  TUTORIAL.  Advise 
then  not  to  use  the  keyboard  and  move/  tha  keyboard  aside. 

5.  After  they  have  completed  the  TUTORIAL,  answer  their  questions  and  give 
them  an  overview  of  how  to  move  through  the  instruction.  First,  they  will  take 
all  the  tests;  then  they  will  take  instruction,  practice,  and  test  for  each 
task.  Re-emphasize  that  they  are  to  notify  you  when  they  are  ready  to  start  a 
block  of  instruction  or  a  test.  Alsc,  tell  then  that  you  need  to  see  the 
screen  providing  feedback  on  the  results  of  eaeh  test.  They  should  not  advance 
past  their  screen  until  you  have  seen  it  and  copied  the  necessary  information. 
Let  them. begir . 


6.  Monitor  students,  recording  their  times,  scores,  and  significant  problems 
on  the  record  sheets.  Be  helpful  but  do  not  over-coach. 

7.  Remember  to  change  videodiscs  after  completion  of  two  remedial  pretests. 

8.  Remember  to  turn  videodisc  over  between  "identify  features/determine 
elevation"  and  other  tasks  during  land  navigation  pretesting  and  training. 
Training  and  te3t3  for  land  navigation  tasks  are  as  follows: 

Land  Nav  Video  Disc 

Side  1 _ (Facing  Down  on  Player)  Side  2 _ (Facing  Down  on  Player) 

Identify  Terrain  Tng  and  Test  Orient  a  Map  Tng  and  Test 

Determine  Elevation  Tng  and  Test  Determine  Location  Tng  and  Test 

Intersection/Resection  Tng  and  Test 

Analyze  Terrain  Trig  and  Test 

9.  Remember  to  change  videodiscs  before  completion  of  two  remedial  posttests. 

10.  If  any  student  is  running  short  on  time,  skip  the  training  and  posttest  on 
"intersection/resection"  or  "analyze  terrain"  or  both,  if  necessary.  If  you 
delete  one,  try  to  alternate  which  one  you  skip  so  that  we  get  roughly  the  same 
number  of  students  on  each.  Don't  work  student  longer  than  8  hours,  not 
counting  the  lunch  break.  Remember:  the  operator  need3  time  to  "back  up"  the 
system. 

11.  Administer  the  posttest  questionnaire  to  each  student.  As  time  allows, 
encourage  them  to  comment  on  the  training  and  record  any  useful  comments  they 
make.  Thank  them  and  send  them  on  their  way. 


INSTRUCTIONS  FOR  STUDENT  RECORD  SHEET  LAND  NAVIGATION  TRIALS 


1-  Enter  student  numbers  at  top  of  sheet.  Student’s  number  for  training  and 
posttest  is  always  100  higher  than  his  pretest  number.  For  example, 
the  first  student  is  #1  for  pretest  and  #101  for  training  and  posttest. 

2.  Provide  student  and  evaluator  with  the  correct  testing  and  training 
SEQUENCE  based  on  ODD  or  EVEN  assigned  student  number. 

3.  Enter  time  started  and  time  ended  each  time  student  starts  or  completes 
training  or  test.  •  Have  students  raise  hands  to  indicate  a  start  or  completion 
Under  Land  Navigation  Training  and  Tests,  enter  2  (two)  start  ard  2  (two)  end 
times,  one  for  training  and  one  for  test.  If  student  fails  to  complete  train¬ 
ing  or  test,  enter  time  stopped. 

k.  Enter  one  of  the  following  codes  for  each  test  result:  G-Go,  N-No  Go, 
I-Incomplete.  Also  enter  number  of  items  correct  and  total  number  of  items 
on  test  under  Test  Score  (e.g.,  G/9  of  12). 

5.  Each  time  a  problem  arises  which  requires  your  intervention,  enter  a  code 
in  the  last  column  indicating  the  nature  of  the  problem.  If  further  explana¬ 
tion  of  the  problem  and  its  solution  is  required,  put  a  number  beside  the  code 
and  write  a  corresponding  footnote  on  a  separate  page.  Codes  are  as  follows: 


Personnel 

SI  -  student  cannot  follow  instructions  for  interacting  with  computer 

SC  -  student  cannot  comprehend  material  presented 

SP  -  student  has  personal  problem  interfering  with  training 

SO  -  other 

SU  -  unknown 


Equipment 

ESH  -  system-wide  (host)  MicroTICCIT  malfunction 

EWS  -  malfunction  in  individual  work-station  processor  or  monitor 

EVD  -  prchlem  with  videodisc  player  or  videodisc  availability 

ETM  -  lack  of  necessary  paper-based  training  materials 

E0  -  other 

EU  -  unknown 
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For  EVEN  numbered  students 


STUDENT  RECORD  SHEET  FOR  LAND  NAVIGATION  TRIALS 
Student  number:  Pretest  #  _  Tng  +  Posttest  it  _ 


Pretest  Postest 

SEQUENCE  G  o-  NG  £Rt/flTrl ed  G  or  NG  #Rt/#Tried  Problem/Solution 

Remedial 


Tank  Positions  (12) 

/ 

/ 

/ 

Visual  Signals  (16) 

/ 

/ 

/ 

Land  Navigation  Pretest 

Analyze  Terrain  (20) 

/ 

/ 

/ 

Intersection  (6) 

/ 

/ 

/ 

Resection  (*0 

Identify  Features  (12) 

/ 

/ 

/ 

Determine  Elevation  (6) 

/ 

/ 

/ 

Remedial 


Tank  Positions 

(12) 

/ 

/ 

/ 

Vi -"al  Signals 

(16)  _ 

/ 

/ 

/ 

Second  Session 

Determine  Location  (6) 


/ 


/ 
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For  ODD  numbered  students 

STUDENT  RECORD  SHEET  FOR  LAND  NAVIGATION  TRIALS 
Student  number:  Pretest  # _ Tng  h-  Posttest  #  _ 


SEQUENCE 

Remedial 

Visual  Signals  (16) 

Pretest 

G  or  NG  #Rt/#TrIed 

/ 

Postest 

G  or  NG  #Rt/#Tried 

* 

/ 

Problem/Solution 

/ 

Tank  Positions  (12) 

/ 

/ 

/ 

Land  Navigation  Pretest 

Identify  Features  (12) 

/ 

/ 

/ 

Determine  Elevation  (6) 

/ 

/ 

/ 

Intersection  (6) 

/ 

/ 

/ 

Resection  (4) 

/ 

/ 

/ 

Analyze  Terrain  (20) 

/ 

/ 

/ 

Remedial 

Visual  Signals  (16) 

/ 

/ 

/ 

Tank  Positions  (12) 

/ 

/ 

/ 

